We first redefine notation for clarity and then provide the proofs of the results in the main paper. Now we first prove that the iteration in Eq.2 has a fixed point. Proof of Lemma 3.1: Let We present the bound on using empirical Bellman operator compared to the true Bellman operator. The proof can be found in [6]. Proof of Theorem 3.4: Recall that the expression of the V -function iterate is given by: Proof of Theorem 3.6: The proof of this statement is divided into two parts.

As aresult, full MOBAgames without restrictions are farfrom being mastered by any existing AI system. In this paper, we propose a MOBA AIlearning paradigm that methodologically enables playing full MOBAgames withdeepreinforcementlearning.Specifically,wedevelopacombinationofnovel and existing learning techniques, including curriculum self-play learning, policy distillation, off-policy adaption, multi-head value estimation, and Monte-Carlo tree-search, intraining andplaying alargepoolofheroes,meanwhile addressing thescalabilityissueskillfully.